Method of entropy randomization on a parallel computer

ABSTRACT

Method, system, and computer program product for randomizing entropy on a parallel computing system using network arithmetic logic units (ALUs). In one embodiment, network ALUs on nodes of the parallel computing system pseudorandomly modify entropy data during broadcast operations through application of arithmetic and/or logic operations. That is, each compute node&#39;s ALU may modify the entropy data during broadcasts, thereby mixing, and thus improving, the entropy data with every hop of entropy data packets from one node to another. At each compute node, the respective ALUs may further deposit modified entropy data in, e.g., local entropy pools such that software running on the compute nodes and needing entropy data may fetch it from the entropy pools. In some embodiments, entropy data may be broadcast via dedicated packets or included in unused portions of existing broadcast packets.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of co-pending U.S. patent applicationSer. No. 13/760,510, filed Feb. 6, 2013. The aforementioned relatedpatent application is herein incorporated by reference in its entirety.

BACKGROUND

Embodiments disclosed herein relate to randomizing entropy on a parallelcomputing system.

High quality random numbers are essential for many aspects of computersystems, most notably in the area of security. However, computer systemshave a difficult time generating high quality random numbers, i.e.,numeric sequences that are close to being truly random. There are manyalgorithms that generate random numbers, but they typically generate thesame sequence of numbers, thus suffering from predictability.

Further, some computing systems may lack, or have limited access to, aconvenient source of entropy. Entropy data from the seemingly randombehavior of a human typing on a keyboard, for example, may beunavailable to computers without keyboard inputs. The computer mayinstead rely on time intervals between interrupts as a source ofentropy, but this is often unhelpful due to the regularity of suchintervals and the ability of outside forces to manipulate suchintervals. As a result, computing systems with limited external or userinputs, such as massively parallel computers or embedded systems, maylack entropy data.

SUMMARY

Embodiments disclosed herein provide a method, system, and computerprogram product for performing an operation for randomizing entropy dataon a parallel computing system, the operation including broadcasting,from a first node of the parallel computing system, first entropy datato one or more nodes of the parallel computing system. The operationfurther includes modifying, by arithmetic logic units (ALUs) on thefirst node and the one or more nodes, the first entropy data during thebroadcast. In addition, the operation includes storing the first entropydata, as modified during the broadcast, at each of the one or morenodes.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF DRAWINGS

So that the manner in which the above recited aspects are attained andcan be understood in detail, a more particular description ofembodiments of the disclosure, briefly summarized above, may be had byreference to the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this disclosure and are therefore not to beconsidered limiting of its scope, for the disclosure may admit to otherequally effective embodiments.

FIG. 1 is a schematic illustrating techniques for randomizing entropy ona massively parallel computing system, according to one embodiment ofthe invention.

FIG. 2 is a block diagram of components of a massively parallel computersystem, according to one embodiment of the invention.

FIG. 3 is a conceptual illustration of a three-dimensional torus networkof the system, according to one embodiment of the invention.

FIG. 4 is a diagram of a node of the system, according to one embodimentof the invention.

FIG. 5 is a flow chart illustrating a method for randomizing entropy ona parallel computing system, according to one embodiment of theinvention.

DETAILED DESCRIPTION

Embodiments of the invention randomize entropy on a parallel computingsystem using network arithmetic logic units (ALUs). In one embodiment,network ALUs on nodes of the parallel computing system pseudo-randomlymodify entropy data during broadcast operations through application ofarithmetic and/or logic operations. That is, each compute node's ALU maymodify the entropy data during broadcasts, thereby mixing, and thusimproving, the entropy data with every hop of entropy data packets fromone node to another. Here, the modification of entropy data may betransparent to kernels and programs on the nodes of the parallelcomputing system and minimize the involvement of such kernels andprograms in managing entropy data so as to reduce noise/jitter whichwould otherwise result from such management. At each compute node, theALUs may deposit modified entropy data in, e.g., local entropy poolssuch that software running on the compute nodes may fetch entropy datafrom the entropy pools. In some embodiments, entropy data may bebroadcast via dedicated packets. Alternatively, entropy data may beincluded in unused portions of existing broadcast packets.

In the following, reference is made to embodiments of the disclosure.However, it should be understood that the disclosure is not limited tospecific described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice thedisclosure. Furthermore, although embodiments of the disclosure mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the disclosure. Thus, the followingaspects, features, embodiments and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the invention” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present disclosure may take theform of an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present disclosure may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present disclosure are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

An entropy property, as used herein, refers to the ability of a randomnumber generator to provide an equal probability of outputting any givenvalue from a set of predefined values that the random number generatoris configured to output. To improve the entropy property of a randomnumber generator, an I/O node or a compute node may add entropy data toits entropy pool, which may include a set of one or more entropy poolelements. Each of the entropy pool elements may include an arbitrarylength sequence of bits, and may be used as input to a random numbergenerator in order to compute a random number.

FIG. 1 is a schematic diagram 100 illustrating techniques forrandomizing entropy on a massively parallel computing system, accordingto one embodiment. As shown, the schematic 100 includes a high-qualitysource of entropy 110, which may be any feasible source that isinternal, or external, to the parallel computing system. Examplesinclude a service node of the parallel computing system having akeyboard on which users type, a hardware random number generator, anexternal file system, correctible error events, and the like.Illustratively, entropy data collected at the entropy source 110 may betransmitted (e.g., periodically) to an input/output (I/O) node 120 ofthe parallel computing system. As discussed in greater detail below, theI/O node 120 may handle I/O operations for a set of compute nodes130-140 of the parallel computing system.

As shown, the I/O node 120 includes a random number generator 121, anentropy pool 122, and an arithmetic logic unit (ALU) 123. In oneembodiment, the random number generator 121 may be configured to useentropy pool elements as initial values in the generation ofpseudorandom numbers. That is, the random number generator 121 maygenerate pseudorandom numbers, with the elements of the entropy pool 122as seed values. To improve the entropy property of the random numbergenerator 121, new entropy data may be added as elements to the entropypool 122 such that more initial values are available for the randomnumber generator 121 to use. In particular, entropy data from thehigh-quality entropy source 110, as modified by the ALU 123, may beadded to the entropy pool 122. As discussed in greater detail below,modification of entropy data during broadcast operations allows the highquality entropy data to undergo pseudorandom variation from node to nodesuch that each node acquires different entropy data.

In one embodiment, the ALU 123 may be a hardware component supporting aninternal network of the parallel computing system, which may facilitatemessage passing. The ALU 123 may be configured to perform arithmetic andlogic operations to, e.g., generate broadcast addresses, manipulatepacket data, and the like. More specifically, the ALU 123 may beconfigured to modify, during a broadcast operation, entropy data beingbroadcasted. For example, during the broadcast, the ALU 123 may performan exclusive or (“XOR”) of the packet's payload (i.e., the entropy data)against some other value to modify the payload pseudo-randomly. Here,the other value may be, e.g., a unique rank or location of the I/O node120, a direction from which the entropy data arrived, other entropy datareceived from the entropy source 110, etc. In one embodiment, thearithmetic or logic operation which is applied may also be determinedbased on, e.g., the node rank or another value. In some embodiments,entropy data may be broadcast via dedicated packets, but entropy datamay also “piggyback” in unused portions of packets broadcast as part ofongoing operations on the computing core.

As shown, packets broadcast by the I/O node 120 may be transmitted tocompute nodes 130 a-b and further forwarded to compute nodes 140 a-b and140 c-d by each of compute nodes 130 a and 130 b, respectively.Illustratively, compute node 130 a includes a random number generator131 a, an entropy pool 132 a, and an ALU 133 a, which may be similar tothe random number generator 121, the entropy pool 122, and the ALU 123,respectively. Compute node 130 b and each of compute nodes 140 a-d mayalso include similar components. In one embodiment, the ALUs 133 a-b andALUs (not shown) of the compute nodes 140 a-d may modify packets havingentropy data payloads via arithmetic and/or logic operation upon e.g.,receipt of the packets, transmittal of the packets, and the like. Doingso mixes, and thus improves, the entropy data with every hop of thepacket from one node to another. As a result, relatively small inputs ofentropy data from the internal or external source may provide abundantentropy data for all nodes in the system. Further, the ALUs may depositthe mixed entropy data in a corresponding entropy pool, from wheresoftware on the respective node may retrieve and use the entropy data.As a result, programs and kernels of the compute nodes 130, 140 may befreed from having to manage entropy data, thereby eliminating noiseand/or jitter which may occur during such management.

FIG. 2 is a block diagram of components of a massively parallel computersystem 200, according to one embodiment disclosed herein.Illustratively, computer system 200 shows the high-level architecture ofan IBM Blue Gene® computer system, it being understood that otherparallel computer systems could be used, and the description of apreferred embodiment herein is not intended to limit the presentdisclosure.

As shown, computer system 200 includes a compute core 201 having anumber of compute nodes arranged in a regular array or matrix, whichperform the useful work performed by system 200. The operation ofcomputer system 200, including compute core 201, may be controlled bycontrol subsystem 202. Various additional processors in front-end nodes203 may perform auxiliary data processing functions, and file servers204 provide an interface to data storage devices such as disk basedstorage 209A, 209B or other I/O (not shown). Functional network 205provides the primary data communication path among compute core 201 andother system components. For example, data stored in storage devicesattached to file servers 204 is loaded and stored to other systemcomponents through functional network 205.

Also as shown, compute core 201 includes I/O nodes 211A-C and computenodes 212A-I. Compute nodes 212 provide the processing capacity ofparallel system 200, and are configured to execute applications writtenfor parallel processing. I/O nodes 211 handle I/O operations on behalfof compute nodes 212. Each I/O node 211 may include a processor andinterface hardware that handles I/O operations for a set of N computenodes 212, the I/O node and its respective set of N compute nodes arereferred to as a Pset. Compute core 201 contains M Psets 215A-C, eachincluding a single I/O node 211 and N compute nodes 212, for a total ofM×N compute nodes 212. The product M×N can be very large. For example,in one implementation M=1024 (1 K) and N=64, for a total of 64 K computenodes. In some embodiments, each of I/O nodes 211 and compute nodes 212may include an ALU similar to the ALUs 123, 133 of FIG. 1, whichimproves entropy data during broadcast operations by applying arithmeticand/logic operations as broadcasted packets hop from one node toanother.

In general, application programming code and other data input requiredby compute core 201 to execute user applications, as well as data outputproduced by the compute core 201, is communicated over functionalnetwork 205. The compute nodes 212 within a Pset 215 communicate withthe corresponding I/O node over a corresponding local I/O collectivenetwork 213A-C. The I/O nodes, in turn, are connected to functionalnetwork 205, over which they communicate with I/O devices attached tofile servers 204, or with other system components. Thus, the local I/Ocollective networks 213 may be viewed logically as extensions offunctional network 205, and like functional network 205 are used fordata I/O, although they are physically separated from functional network205. One example of the collective network is a tree network. Asdiscussed in greater detail below, an I/O node 211 may broadcast entropydata over such a tree network, and network ALUs of each compute node 212on the tree network may pseudorandomly modify the entropy data duringthe broadcast operation. That is, during each hop of entropy datapackets from one compute node 212 to another in the tree network, theALUs of the compute nodes 212 may, e.g., upon receipt or transmission ofthe packets, modify the entropy data. Entropy data, as modified duringthe broadcast operations, may then be stored at each of the computenodes 212 and used by applications running thereon (e.g., random numbergenerators).

Control subsystem 202 directs the operation of the compute nodes 212 incompute core 201. Control subsystem 202 is a computer that includes aprocessor (or processors) 221, internal memory 222, and local storage225. An attached console 207 may be used by a system administrator orsimilar person. Control subsystem 202 may also include an internaldatabase which maintains state information for the compute nodes in core201, and an application which may be configured to, among other things,control the allocation of hardware in compute core 201, direct theloading of data on compute nodes 211, and perform diagnostic andmaintenance functions.

Control subsystem 202 communicates control and state information withthe nodes of compute core 201 over control system network 206. Network206 is coupled to a set of hardware controllers 208A-C. Each hardwarecontroller communicates with the nodes of a respective Pset 215 over acorresponding local hardware control network 214A-C. The hardwarecontrollers 208 and local hardware control networks 214 are logically anextension of control system network 206, although physically separate.

In addition to control subsystem 202, front-end nodes 203 providecomputer systems used to perform auxiliary functions which, forefficiency or otherwise, are best performed outside compute core 201.Functions which involve substantial I/O operations are generallyperformed in the front-end nodes. For example, interactive data input,application code editing, or other user interface functions aregenerally handled by front-end nodes 203, as is application codecompilation. Front-end nodes 203 are connected to functional network 205and may communicate with file servers 204.

In one embodiment, the computer system 200 determines, from among aplurality of class route identifiers for each of the compute nodes alonga communications path from a source compute node to a target computenode in the network, a class route identifier available for all of thecompute nodes along the communications path. The computer system 200configures network hardware of each compute node along thecommunications path with routing instructions in dependence upon theavailable class route identifier and a network topology for the network.The routing instructions for each compute node associate the availableclass route identifier with the network links between that compute nodeand each compute node adjacent to that compute node along thecommunications path. The source compute node transmits a network packetto the target compute node along the communications path, which includesencoding the available class route identifier in a network packet. Thenetwork hardware of each compute node along the communications pathroutes the network packet to the target compute node in dependence uponthe routing instructions for the network hardware of each compute nodeand the available class route identifier encoded in the network packet.As used herein, the source compute node is a compute node attempting totransmit a network packet, while the target compute node is a computenode intended as a final recipient of the network packet.

In one embodiment, a class route identifier is an identifier thatspecifies a set of routing instructions for use by a compute node inrouting a particular network packet in the network. When a compute nodereceives a network packet, the network hardware of the compute nodeidentifies the class route identifier from the header of the packet andthen routes the packet according to the routing instructions associatedwith that particular class route identifier. Accordingly, by usingdifferent class route identifiers, a compute node may route networkpackets using different sets of routing instructions. The number ofclass route identifiers that each compute node is capable of utilizingmay be finite and may typically depend on the number of bits allocatedfor storing the class route identifier. An “available” class routeidentifier is a class route identifier that is not actively utilized bythe network hardware of a compute node to route network packets. Forexample, a compute node may be capable of utilizing sixteen class routeidentifiers labeled 0-15 but only actively utilize class routeidentifiers 0 and 1. To deactivate the remaining class routeidentifiers, the compute node may disassociate each of the availableclass route identifiers with any routing instructions or maintain a listof the available class route identifiers in memory.

Routing instructions specify the manner in which a compute node routespackets for a particular class route identifier. Using different routinginstructions for different class route identifiers, a compute node mayroute different packets according to different routing instructions. Forexample, for one class route identifier, a compute node may routepackets specifying that class route identifier to a particular adjacentcompute node. For another class route identifier, the compute node mayroute packets specifying that class route identifier to differentadjacent compute node. In such a manner, two different routingconfigurations may exist among the same compute nodes on the samephysical network.

In one embodiment, compute nodes 212 are arranged logically in athree-dimensional torus, where each compute node 212 may be identifiedusing an x, y and z coordinate. FIG. 3 is a conceptual illustration of athree-dimensional torus network of system 200, according to oneembodiment disclosed herein. More specifically, FIG. 3 illustrates a4×4×4 torus 301 of compute nodes, in which the interior nodes areomitted for clarity. Although FIG. 2 shows a 4×4×4 torus having 64nodes, it will be understood that the actual number of compute nodes ina parallel computing system is typically much larger. For example, acomplete Blue Gene/L system includes 65,536 compute nodes. Each computenode 212 in torus 301 includes a set of six node-to-node communicationlinks 302A-F which allows each compute nodes in torus 301 to communicatewith its six immediate neighbors, two nodes in each of the x, y and zcoordinate dimensions.

As used herein, the term “torus” includes any regular pattern of nodesand inter-nodal data communications paths in more than one dimension,such that each node has a defined set of neighbors, and for any givennode, it is possible to determine the set of neighbors of that node. A“neighbor” of a given node is any node which is linked to the given nodeby a direct inter-nodal data communications path. That is, a path whichdoes not have to traverse another node. The compute nodes may be linkedin a three-dimensional torus 301, as shown in FIG. 2, but may also beconfigured to have more or fewer dimensions. Also, it is not necessarilythe case that a given node's neighbors are the physically closest nodesto the given node, although it is generally desirable to arrange thenodes in such a manner, insofar as possible.

In one embodiment, the compute nodes in any one of the x, y or zdimensions form a torus in that dimension because the point-to-pointcommunication links logically wrap around. For example, this isrepresented in FIG. 3 by links 302D, 302E and 302F which wrap aroundfrom a last node in the x, y and z dimensions to a first node. Thus,although node 303 appears to be at a “corner” of the torus, node-to-nodelinks 302A-F link node 303 to nodes 302D, 302E and 302F, in the x, y andz dimensions of torus 301.

FIG. 4 is a diagram of an I/O node 211 of the system 200 of FIG. 2,according to one embodiment. The I/O node 211 may correspond to the I/Onodes 110 and 111 of FIG. 1. In one embodiment, each I/O node 211 may bephysically implemented as a single, discrete integrated circuit chip.

As shown, I/O node 211 includes processor cores 401A and 401B, and alsoincludes memory 402 used by both processor cores 401; an externalcontrol interface 403 which is coupled to local hardware control network214; an external data communications interface 404 which is coupled tothe corresponding local I/O collective network 213, and thecorresponding six node-to-node links 302 of the torus network 301; andmonitoring and control logic 405 which receives and responds to controlcommands received through external control interface 403. Monitoring andcontrol logic 405 may access processor cores 401 and locations in memory402 on behalf of control subsystem 202 to read (or in some cases alter)the operational state of I/O node 211.

In a Blue Gene system, the external data interface 404 may transmitmessage passing interface (MPI) messages by encapsulating such messageswithin packets and transmitting the packets of over a network (e.g., atree or torus network). For example, the MPI call of MPI_Send( ) may beused to transmit a message, and the call of MPI_Bcast( ) may be used tobroadcast the message. Other parallel systems and other parallelcomputing libraries may include similar mechanisms. Illustratively, theexternal data interface 404 includes an ALU 404 a which may be similarto the ALUs 123, 133 discussed above with respect to FIG. 1. That is,ALU 404 a may be configured to modify, during a broadcast operation,entropy data being broadcasted. ALU 404 a may generally be capable ofperforming arithmetic and logic operations on packets traversing theexternal data interface 404. More specifically, ALU 404 a may performoperations (e.g., an XOR operation against some value) to pseudorandomlymodify entropy data in packets being broadcast to compute nodes (notshown), thereby mixing and improving the entropy data.

As shown in FIG. 4, memory 402 stores an operating system image 411, anapplication code image 412 and user application data structures 413 asrequired. Some portion of memory 402 may be allocated as a file cache414, i.e., a cache of data read from or to be written to an I/O file.Operating system image 411 provides a copy of a simplified-functionoperating system running on I/O node 211. Operating system image 411 mayinclude a minimal set of functions required to support operation of theI/O node 211. As shown, the memory 402 also contains entropy pool 412.In some embodiments, the entropy pool may be similar to the entropypools 122, 132 of FIG. 1. As discussed with reference to FIG. 1, theentropy pool 412 is generally configured to store entropy pool elementsreceived from an internal or external entropy source, which, asdiscussed, may be a service node, a hardware random number generator,correctible error events, and the like. The pool elements may then bedistributed to a random number generator 416 for use in creatingpseudorandom numbers, and distributed via broadcast operations tocompute nodes. As discussed, the ALU 404 a may perform arithmetic and/orlogic operations to modify entropy data during such broadcast operationsso that relatively small inputs of entropy data from the internal orexternal source may provide abundant entropy data for all nodes in thesystem.

Although discussed in FIG. 4 with respect to an I/O node, compute nodes212 of the parallel computing system may also have processors, memories,external data interfaces, etc. Further, the external data interfaces ofthe compute nodes may each include an ALU which, similar to ALU 404 a,performs arithmetic and/or logic operations to modify entropy dataduring broadcast operations. Doing so mixes, and thus improves, theentropy data with every hop of the packet from one node to another.

FIG. 5 is a flow chart illustrating a method 500 for randomizing entropyon a parallel computing system, according to one embodiment. As shown,the method 500 begins at step 510, where an I/O node of the parallelcomputing system receives entropy data from an internal or externalsource. Any feasible internal or external entropy source may be used.For example, the source may be a service node having a keyboard on whichusers type, a hardware random number generator, an external file system,correctible error events, and the like. In one embodiment, the sourcemay transmit entropy data periodically, but not so frequently as todeplete the source itself. Entropy data received from the internal orexternal source, which may be stored as elements in an entropy pool onthe I/O node, is then modified during broadcast operations. As a result,relatively small inputs of entropy data from the source may provideabundant entropy data for all nodes in the system.

At step 520, the I/O node broadcasts entropy pool elements to computenodes in a tree network. For example, in a Blue Gene system, the MPIcall of MPI_Beast( ) may be used to broadcast messages including suchentropy pool elements over the tree network. Other parallel systems andother parallel computing libraries may use similar mechanisms forbroadcasting messages. In one embodiment, the I/O node may periodicallybroadcast entropy pool elements such that each of the compute nodes hassufficient entropy data. As discussed, dedicated packets may be used forsuch broadcasts. Alternatively, unused space within normal networkpackets may be used.

At step 530, network ALUs pseudorandomly modify the entropy data as ittraverses the tree network. As discussed, an ALU of the I/O node mayinitially perform arithmetic and/or logic operation(s) to pseudorandomlymodify entropy data leaving the I/O node. ALUs of compute nodes mayfurther modify the entropy data during each hop of the entropy datapackets from one node to another, thereby mixing, and thus improving,the data during the broadcast. For example, the network ALUs may modifyreceived entropy data packets before depositing those packets in a localentropy pool, and also forward those modified packets to other computenodes. The network ALUs may use any feasible arithmetic and/or logicoperation(s) to modify the broadcast entropy data. For example, eachnetwork ALU may perform an XOR of the entropy data against anothervalue, such as a rank or location of the node of the ALU, a directionfrom which the entropy data arrived, other entropy data, and the like.Of course, other arithmetic and/or logic operations (e.g., addition,AND, OR, etc.) against other values may be performed to modify theentropy data. In one embodiment, the particular arithmetic or logicoperation which is applied may also be determined based on, e.g., thenode rank or another value.

At step 540, the entropy data, as modified by network ALUs during thebroadcast operation, is stored at the compute nodes. For example, aftermodification by an ALU of a compute node, received entropy data (whichmay also have been modified during previous hops) may be stored aselements of an entropy pool located in a memory of the compute node. Ofcourse, the entropy data may also be stored elsewhere, such as inregisters in the ALU, in a storage device, etc. Applications which needentropy data may then fetch (e.g., by invoking an applicationprogramming interface (API)) the stored entropy data for use. Forexample, a random number generator may fetch entropy data to use asinitial values in producing pseudorandom numbers.

Although discussed above primarily with respect to broadcasts over atree network by an I/O node, similar techniques may be used to modifyentropy data packets broadcast over networks having other topologies,and other nodes may perform the broadcast. Further, in some embodiments,network ALUs may modify entropy during operations other than broadcastoperations. In one embodiment, entropy data may bounce around betweennodes endlessly, being modified by network ALUs during the bounces. Inanother embodiment, applications on compute nodes may invoke an API torequest entropy data from the I/O node. In such a case, network ALUs maypseudorandomly modify entropy data transmitted by the I/O node inresponse to the request as the entropy data traverses the network. Inaddition, although discussed above primarily with respect to networkALUs, ALUs present elsewhere may also modify entropy data according totechniques disclosed herein.

Advantageously, embodiments disclosed herein modify entropy databroadcast over a parallel computer network to provide distinct entropydata to computing node within the computer network. Doing so improvesthe quality of the entropy data traversing nodes of the parallelcomputer, and also ensures that the entropy data on each of the nodes isdifferent. Further, relatively small inputs of original entropy datafrom an internal or external source may provide abundant entropy datafor all nodes in the system. In one embodiment, network ALUs perform thepseudorandom modifications, thereby freeing the kernels and programs ofthe parallel computer nodes from noise/jitter due to having to manageentropy. Using entropy data, as improved by techniques disclosed herein,random number generators of the compute nodes may have higher entropyproperties, such that the random number generators have a more equalprobability of outputting any given value from a set of predefinedvalues.

While the foregoing is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A method for randomizing entropy data in aparallel computing system, comprising: broadcasting, from a first nodeof the parallel computing system, first entropy data to one or morenodes of the parallel computing system; modifying, by arithmetic logicunits (ALUs) on the first node and the one or more nodes, the firstentropy data during the broadcast upon at least one of a receipt and atransmittal of packets which include the first entropy data by the firstnode and the one or more nodes; and storing, at each of the one or morenodes, the first entropy data, either as received or as received andmodified by the respective node.
 2. The method of claim 1, wherein thefirst entropy data is broadcast over a tree network, and wherein eachnode modifies the received first entropy data, stores the first entropydata as modified by the respective node, and sends the first entropydata as modified by the respective node to child nodes.
 3. The method ofclaim 1, wherein the first node is an input/output (I/O) node whichprovides I/O services in a tree network to a set of compute nodes. 4.The method of claim 1, wherein the ALUs modify the first entropy data byperforming at least one of an arithmetic and a logic operation.
 5. Themethod of claim 4, wherein the at least one of the arithmetic and thelogic operation is performed against one of a node rank, a nodelocation, and a direction from which the first entropy data that isreceived arrived at the respective node.
 6. The method of claim 4,wherein the at least one of the arithmetic and the logic operation isperformed against second entropy data received from the first node. 7.The method of claim 1, wherein the first entropy data is broadcast usingeither dedicated packets or unused space of non-dedicated packets. 8.The method of claim 1, wherein the first entropy data is received by thefirst node from an entropy source external to the parallel computingsystem.
 9. The method of claim 1, further comprising, generating apseudorandom number at least one of the one or more nodes using, as aninitial value, the modified first entropy data stored at that node.